Skip to content

Hosted GPU tier: endpoint, launcher, reaper, runner image, clients (GJOB-088) - #1

Closed
arvandnavabi wants to merge 67 commits into
mainfrom
main-merged
Closed

arvandnavabi wants to merge 67 commits into
mainfrom
main-merged

Conversation

@arvandnavabi

Copy link
Copy Markdown

Everything behind api.forcefieldsilicon.com and mdengine run --gpu, built 2026-09-03 → 09-06. Merge commit over unrelated histories (local lineage diverged after v0.6.1); merged tree == hosted-2026-09-06, zero conflicts.

  • hosted/endpoint: stdlib service (hashed API keys, credits ledger, /welcome instant-key page + Stripe webhook, job state machine, RunPod REST v2 launcher with ladder fallback, launch timeout, reaper), admin CLI, Caddy/systemd/bootstrap/deploy, hourly sqlite backup timer — 33 tests
  • docker/runner-gpu: pull-runner, pod-side TTL, Dockerfile.prebuilt + GHCR workflow → ghcr.io/forcefieldsilicon/mdengine-runner-gpu:ADA89
  • Clients: HostedClient in LAMMPSCore; CLI run --gpu / login / account / jobs; MCP host=cloud; app "Run accelerated"
  • hosted/CONTRACT.md: purchase flow + pod-lifecycle invariant

🤖 Generated with Claude Code

arvandar and others added 30 commits August 31, 2026 10:51
macOS MD workbench: Metal trajectory viewer (orbit/pan/zoom, timeline
scrubber), mdengine CLI (info/export/decimate/run), and an MCP stdio server
with a detached LAMMPS job runner. Fresh public-history repo; internal
development history stays in the private working repo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ld Silicon branding

- Scrubber shows frame numbers under major ticks; major/minor grid spacing
  user-selectable (10/20/25/50% and 1/2/5/10%)
- Right-side inspector (View menu, ⌥⌘I, or sidebar button): timeline grid,
  atom size, background, orbit speed, camera reset, element legend — live via
  the same UserDefaults keys as Settings
- Brand: launching under ForceField Silicon (Gitinama Inc. trade name);
  bundle id com.forcefieldsilicon.mdengine; README/LICENSE/manual updated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t agent-security note

- Bundle fe_oxidation.in/.xyz: bcc Fe slab + O2 gas via ffield.reax.Fe_O_C_H
  (ships with LAMMPS - nothing redistributed); 391 atoms, 13 frames, runs in
  ~17s on 6 omp threads. First step of the metal-oxidation focus; Al/O ReaxFF
  pending a force-field licensing check.
- CLI + MCP derive LAMMPS_POTENTIALS from the lmp install when unset, so decks
  can reference bare force-field names out of the box (verified end-to-end).
- README: decks-are-programs security note now states plainly that an agent on
  the MCP server can submit decks that execute with user privileges; sandboxing
  is scoped to the hosted tier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- LammpsDumpParser: ITEM: TIMESTEP blocks from dump atom/custom; direct,
  unwrapped, and scaled (box-mapped) coordinates; element column when present,
  numeric type token otherwise; rows sorted by id. TrajectoryReader sniffs
  dump-vs-XYZ, and every reader (app Load File, CLI info/export/decimate, MCP
  trajectory tools) now goes through it. Verified: scaled and element dump
  variants parse bit-identical bboxes to the XYZ reference.
- Unknown element tokens (numeric types) get stable distinct hash colours.
- docker/runner: two-stage LAMMPS stable_29Aug2024 OpenMP build, non-root,
  potentials baked in; README documents the hardening flags that constitute
  the sandbox (no network, read-only, resource caps).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…robustness

From peer-session shakedown on real oxidation dumps (RUN-010, id type x y z q):
- Reported misparse (element=id, shifted coords, wrong counts) was a STALE
  RELEASE BINARY on PATH, not parser logic - release now rebuilt with every
  change; parser verified against 150-frame growing-count dump (8402->8421).
- Real fixes/features: -h/--help on subcommands (was 'cannot read --help');
  info prints the dump's per-atom field list and charge range; Arv carries q
  from native dumps; export/decimate --charges writes extended-XYZ
  (Properties=species:S:1:pos:R:3:charge:R:1); --elements O,Al maps numeric
  type tokens to symbols; rows with non-finite coords dropped (NaN-poisoned
  RUN-006 tail frames now parse with finite bbox). MCP: trajectory_info gains
  fields/charges, export_frame gains charges option.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dump)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Tests/AppTests: 11 XCTest cases over both parsers, reader sniffing, and the
  extended-XYZ writer (scaled coords, charges, id sort, growing counts,
  truncated in-flight tails, NaN drops); test target fixed to match its dir
  so 'swift test' actually runs it. All green.
- MCP: new job_files tool (locate a finished job's outputs); refuse >2GB
  trajectory loads with actionable guidance (CLI too); serverInfo 0.5.0.
- README: full MCP tool table, Claude Desktop registration, platform/limits.
- GitHub Actions CI (build + test + release build on macos-14), activates on
  first push.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… scaffold

- Double-clicking .xyz/.extxyz/.lammpstrj/.traj/.dump now opens in MDEngine
  (CFBundleDocumentTypes + AppDelegate open-URL buffering for launch races)
- scripts/make_dmg.sh packages dist/MDEngine-<version>.dmg with /Applications
  symlink; make_app.sh signs with DEVELOPER_ID + notarizes/staples when
  NOTARY_PROFILE is set, ad-hoc otherwise - Apple-day is config-only
- Version 0.5.0 across app + MCP

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Play/pause button beside the timeline with a Loop checkbox beneath it
  (checked = repeat from frame 1 when playback reaches the end; persisted)
- Live following: a loaded file is polled every 2s; frames appended by a
  running simulation extend the timeline automatically (green 'live' badge).
  Camera and scrub position are preserved; if the user was at the final frame
  the view follows the new final frame. Skipped above 512MB.
- Gesture terminology unified as Orbit / Pan / Zoom / Reset — window hint,
  manual, and code comments now all say 'Pan: double-click-drag'

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Root cause (hung on a 343MB growing oxidation dump): load() parsed on the
main thread, then live-follow re-parsed the WHOLE file every 2s, also on the
main thread. Fixes:
- All parsing moved to a background queue; UI shows 'Loading …' and stays
  responsive; results published on main
- Live-follow: in-flight guard (never overlapping parses), poll interval
  scales with file size (~45s at 350MB), 1GB follow cap, watch invalidated
  if a different file is loaded meanwhile
- App-side 2GB open guard with decimate guidance (CLI/MCP already had one)
- Arv drops its unused Identifiable UUID (millions of pointless allocations
  per parse); renderer builds GPU atom arrays per frame on demand instead of
  precomputing every frame (~350MB saved on 1276-frame trajectories)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Window scene replaces WindowGroup: macOS restoration was resurrecting blank
  duplicate windows; every window shared one trajectory anyway
- View menu + inspector View section: Perspective/Orthographic toggle (⌥⌘P).
  Ortho frames the height perspective would see at the current distance, so
  zoom works and switching holds framing; point size pre-divided by distance
  to match apparent atom size
- Window title shows the loaded file name

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Root cause (seen twice in screenshots: restored windows rendering window-bg
gray): AppKit recreates the MTKView while the representable's Coordinator -
and its uploaded-generation counter - survives. The fresh Renderer never
received the trajectory and every draw early-returned without painting.
makeNSView now resets the coordinator's generation so the next update
re-uploads, and draw() always encodes a clear pass so an atomless view shows
the intentional background instead of never painting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MTKView's internal display link silently never started for re-hosted windows
(process sample showed zero draw calls while UI stayed live). The view now
pauses the internal loop and owns a CADisplayLink created in
viewDidMoveToWindow — rendering runs exactly while the view is in a window.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tracked on every successful load (deduped, most recent first, persisted).
Menu shows filenames with full-path tooltips; stale entries are pruned with
an explanatory alert; Clear Menu resets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- ZProfileAnalysis in LAMMPSCore: surface plane (top-5% substrate mean z),
  probe penetration depths, at-surface/in-flight counts, bound-probe mean
  charge, relative-z histogram; unit-tested; shared by app and MCP.
- Inspector: Z-profile section (substrate/probe pickers, live stats, mini
  histogram) + Scale bar toggle in the View section.
- Viewport scale bar: round 1/2/5x10^n A length, driven by camera distance
  and trajectory normalization via ViewportScale; exact at center depth.
- MCP: z_profile tool (frame/substrate/probe args, defaults to the two most
  abundant elements).
- Manual: Inspector analysis section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persisted as playbackFPS; retimes a running playback live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… core

- New MDRender target: RenderCore (one shader/camera-math/palette shared by
  the interactive view and offscreen rendering), OffscreenRenderer, and
  VideoExporter (AVFoundation H.264 MP4 + ImageIO animated GIF).
- Annotations (scale bar + frame counter) baked by default, toggleable.
- Cinematic orbit option; sprite sizes scale with output resolution so 4K
  exports match the window (maxPointSize uniform added to the shader).
- App: File > Export Video... (shift-cmd-E) + Inspector Video-export section
  (resolution/fps/stride-with-auto/annotations/orbit, progress + cancel).
- Viewport View menu snaps to canonical Top/Bottom/Front/Rear (z-up rig);
  Inspector panes 2-4 add extra fixed-view viewports in a CAD-style grid.
- MCP: render_video tool (camera in degrees, stride, orbit, annotations,
  type->element mapping via 'elements'); element mapper moved to LAMMPSCore.
- Manual: Views + Video export sections.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed state)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… viewport View menu

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… view picker

Pane rows show a status summary (view name / Off) collapsed; pane 1 is the
main viewport (always on, Free + snap presets synced with the View menu).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pans

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…deo export second-to-last

LazyVGrid collapsed rows after the first because Metal views have no
intrinsic size; the viewport grid now uses explicit HStack rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… in Restore Defaults

Home/reset camera = isometric (corner view, direction cosines 135/45/-45 to
the axes). Camera rig gains a roll axis so Left keeps z up. Restore Defaults
now resets pane 1-4 (Isometric main; 2/3/4 = Top/Left/Front, off).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bundled example now waits 0.4 s and yields to any real open event
(sourceName set = a load is underway, frames set = one landed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…able)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…kbox overrides

Each extra pane now has its own camera publisher so its bar tracks its own
zoom; checkbox style distinguishes scale-bar controls from pane switches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/Left)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
arvandar and others added 27 commits September 1, 2026 19:51
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restore Defaults / Reset Camera bumps one token for all panes; extra panes
now re-apply their assigned preset after the reset instead of inheriting
the main pane's isometric home.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… renders

- VideoExporter.exportPNG: single frame -> PNG, same camera/style/annotation
  pipeline as video. contrastStyle(): minority species enlarged 1.8x and
  recolored to pop against the substrate.
- render_video and render_image accept colors ('O=red,Al=#3366ff' — named or
  hex), sizes ('O=1.8'), style: 'contrast'; shared arg parsing.
- README/manual updated with the new tools.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gle in the toolbar; signed DMG + tools tarball scripts

- Inspector ▸ Z-profile: numeric depth axis (Å rel. surface), labeled surface-plane
  marker, chart tooltip; Export CSV… / Export Excel… of the LAST frame
  (LAMMPSCore/ZProfileExport: sectioned CSV, dependency-free two-sheet .xlsx)
- Inspector toggle moved from the status bar to a full-size toolbar button
  (top-right, "Hide or show the Inspector"), matching Xcode
- make_dmg.sh signs + notarizes + staples the DMG when DEVELOPER_ID/NOTARY_PROFILE
  are set; new make_tools.sh packages signed CLI + MCP binaries
- version 0.6.1 (bundle + MCP serverInfo)
…host=, list_hosts, fetch_job

Same job contract, different machine: deck dir rsynced up (trajectories/
checkpoints/logs excluded), LAMMPS launched under nohup with the exit code
recorded remotely; job_status/job_log/job_files/cancel_job read state over ssh;
fetch_job pulls the run directory back for the analysis and render tools.
A GPU host is a launch template with the KOKKOS flags. Explicit 'cancelled'
state for local and remote jobs. Verified end to end against ssh localhost.
…en MDE_JOB_ID is set, sshd dev mode otherwise
…fline e2e verified (submit→pull→run→upload→done→results→debit)
…login/account/jobs/job; MCP host=cloud across all job tools; App Run Accelerated + Accelerated Runs window + Settings tab; results auto-open. Verified vs mock from all three surfaces; 20 tests

Co-Authored-By: Claude Code <noreply@anthropic.com>
…nt-key /welcome + Stripe webhook, admin CLI, Caddy/systemd/bootstrap/deploy, 19 tests; CONTRACT: purchase flow + pod lifecycle
…llback, launch timeout, pod release on every terminal state; 33 tests; untrack pycache
…s — Community cudaErrorUnknown was undiagnosable
…under timeout(wall+600) — CONTRACT pod-lifecycle enforcer 3; hourly sqlite backup timer
@arvandnavabi

Copy link
Copy Markdown
Author

Superseded by 8c46a38 on main: the 0.6.3 snapshot with the hosted MCP remote and OAuth 2.1 is now the default branch.

@arvandnavabi
arvandnavabi deleted the main-merged branch September 9, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants